Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added HEC error messages #36871

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

sbylica-splunk
Copy link

Description

Errors returned from Splunk HEC will be displayed in logs. This will help discover some issues without extensive debugging.

Testing

Unit testing and manual testing of the feature

@sbylica-splunk
Copy link
Author

Had to close the previous PR because of messy commit history, so reopening here.

#35712

Copy link
Contributor

github-actions bot commented Jan 1, 2025

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Jan 1, 2025
@sbylica-splunk sbylica-splunk requested a review from atoulme January 8, 2025 14:15
@dmitryax
Copy link
Member

dmitryax commented Jan 9, 2025

Let's resolve the discussion #35712 (comment) first

@github-actions github-actions bot removed the Stale label Jan 9, 2025
@sbylica-splunk
Copy link
Author

Let's resolve the discussion #35712 (comment) first

Hi @dmitryax, I responded to the discussion you linked, any updates?

Copy link
Contributor

github-actions bot commented Feb 4, 2025

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Feb 4, 2025
Copy link
Contributor

Closed as inactive. Feel free to reopen if this PR is still being worked on.

@github-actions github-actions bot closed this Feb 18, 2025
@dmitryax dmitryax reopened this Feb 18, 2025
@dmitryax dmitryax removed the Stale label Feb 18, 2025
@dmitryax
Copy link
Member

Sorry for the delay. My goal was to avoid recreating the body. I’ll try to quickly prototype my suggestion

unmarshalError := json.Unmarshal(body, &jsonResponse)

if unmarshalError == nil {
responseErrorValue, ok := jsonResponse["text"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is being used by the Signalfx exporter, not only the Splunk HEC exporter. The SignalFx exporter probably doesn't return JSON with the text field, so it'll be a redundant step in that add an unnecessary error message from Splunk found to all errors.

Later in this function, we dump the whole payload for http.StatusBadRequest and http.StatusUnauthorized. Why can't we just keep doing that? If you want to extend the same behavior for some other error codes that are expected to go along with the body, feel free to do that

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with that is that httputil.DumpResponse only returns a generic error - like HTTP 403 Forbidden. If we want to display messages returned by splunk (https://docs.splunk.com/Documentation/Splunk/9.2.0/Data/TroubleshootHTTPEventCollector#Possible_error_codes) we have to actually read the body.

For the unnecessary message, I guess it would make sense to just skip this condition then?

@sbylica-splunk sbylica-splunk requested a review from dmitryax March 6, 2025 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants